The reliability of patients' judgements of care in general practice: how many questions and patients are needed?

OBJECTIVES To estimate the number of questions and patients that are needed to achieve reliable measurements of patients' judgements of care in general practice. DESIGN Sensitivity study, using generalisibility theory and real data from surveys of patients. SUBJECTS 739 patients with chronic illness from 23 general practitioners in The Netherlands. MAIN MEASURES The reliability coefficients of scores per patient and scores per general practitioner for patients' judgements of nine dimensions of care in general practice. RESULTS For most dimensions the reliability per patient was 0.80 or higher if three questions were used, but for the evaluation of the "organisation of appointments" and "premises" five questions had to be used. To reach a reliability coefficient of 0.80 per general practitioner three questions and 90 patients, or five questions and 60 patients, were needed for most dimensions. Even more patients or questions were needed for the dimensions "availability for emergencies", premises, and "continuity". A reliability of 0.70 per general practitioner could be achieved if three questions and 60 patients were used, except for availability for emergencies and premises, for which more patients or questions were required. CONCLUSIONS Surveys of patients can only provide reliable information if the samples of questions and patients are large enough. It is important to distinguish between the reliability of scores per patient and the reliability per care provider, as well as between different dimensions of care. The reliability per patient is good for most dimensions if three questions are used, but a good reliability per care provider requires more questions or patients.